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Abstract — In source coding, either with or without side infor- 
mation at the decoder, the ultimate performance can be achieved 
by means of random binning. Structured binning into cosets 
' of performing channel codes has been successfully employed in 
, practical applications. In this letter it is formally shown that 
. various convolutional- and turbo-syndrome decoding algorithms 
proposed in literature lead in fact to the same estimate. An 
$^ ' equivalent implementation is also delineated by directly tackling 
l_ syndrome decoding as a maximum a posteriori probability 
. problem and solving it by means of iterative message-passing. 
■ This solution takes advantage of the exact same structures and 
' algorithms used by the conventional channel decoder for the code 
t-H \ according to which the syndrome is formed. 

Index Terms — Slepian-Wolf coding, source coding, syndrome- 
based binning, turbo codes, message-passing. 
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Fig. 1. A portion of the factor-graph used for decoding (a) convolutional 
codes and (b) systematic syndromes. 
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I. Introduction 

Several practical schemes for lossless source coding with 
side information at the decoder directly embody the optimal 
binning scheme given by Wyner in JT]. In practice, the syn- 
drome with respect to a turbo (or an LDPC) code that is a good 
channel code for the fictitious correlation channel between 
the source and the side information is used as compressed 
representation of a given realization. At the decoder the best 
estimate is found into the coset signalled by this syndrome. 

In this letter, the focus is on the schemes based on convo- 
lutional and (parallel) turbo codes (2), 0, HU, 0. All these 
approaches use in fact a systematic syndrome former as source 
encoder, since this is a computationally efficient solution for 
syndrome formation w.r.t. these codes, in particular w.r.t. turbo 
codes. Decoding is instead performed with ad-hoc algorithms. 
Despite this, here it is shown that the resulting estimate is the 
same in all cases and could be found by simply performing a 
maximum a posteriori probability (MAP) search. 

II. Channel Coding with Convolutional and 
Turbo Codes 

In channel coding based on an (n, k) convolutional code 
over GF(q), encoding is often realized by the systematic 
generator G(D) = [Ik\P(D)]. The actual structure of the 
trellis section that realizes the fc-in (n — fc)-out system P(-D) 
is described by the function X^!_ 1 (xf, xf), which indicates 
if at any time i the transition from state <Tj_i to state Uj is 
possible with the systematic fc-tuple xf as input and the parity 
(n — fc)-tuple x P as output. 

Manuscript received February 17, 2010. The associate editor coordinating 
the review of this letter and approving it for publication was Prof. Z. Yan. 

L. Cappellari is with the Dept. of Information Engineering, University of 
Padova, 35131 Padova, Italy (email: lorenzo.cappellari@dei.unipd.it). 



The symbol-by-symbol MAP decoder evaluates 
argmax x s p (xf|y s (D), y p (D)), where y s (D) and y p (D) 
are the received sequences corresponding to the systematic 
and to the parity sequence, respectively, sent by the encoder 
(formal vector power series are defined as x(Z?) = ~X-iD l , 
where xj are row vectors). If the transmission channel is 
memoryless, i.e. is defined by p(yf |x|) (and by p(yf |x?)), 
and the incoming symbols into the encoder are Ltd., 
p(xf |y s (D),y p (Z?)) can be efficiently found using the 
BCJR algorithm, that is by means of message-passing 
marginalization over the factor-graph shown (according 
to the convention of [6|) in Fig. |l(a)| For example, at 
the i-th step of the forward phase the input messages 
Mi( x D =P( x !)p(y||x|) ocp(xf|yf) and ^(x?) = p(yf |x? ) 
are used to evaluate the forward metric 

ati{o-i) =^XaJ_i( x i' x ?) Q! i-l( ''- 1 )Mi(x|)Mi(Xi) 

(52~x indicates the sum w.r.t. all involved variables except x). 
Once also the backward metric /3i(<7j) is available, the optimal 
choice at time i is found maximizing jUj(xf )A,-(xf ), where 

A,(x|) = ^ X ^ 1 (x|,xf)a i _ 1 ( f 7 < _ 1 )A(a i ) Mi (x|') . 

In case of turbo coding, two (rij , k) systematic convolu- 
tional encoders are used (j = 0, 1). MAP decoding is approx- 
imated with an iterative procedure that alternatively decodes 
the two received parity sequences. The input messages for 
decoding one code are taken from the output messages relative 



to the other one; in particular, jj^p (xf ) oc p(x| |y| 31 (xf ). 

III. Syndrome-Based Source Coding 

Given a linear code, any corresponding parity-check matrix 
defines a source encoder as it bins any source sequence 
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according to its syndrome. In correspondence of an (n, k) con- 
volutional code, the systematic parity-check matrix H(_D) = 
[— P T (D)|I„„fc] represents the most straightforward choice 
for this purpose. In fact, if the source sequence (of i.i.d. n- 
tuples over GF(q)) x(Z3) is broken into the sequences x s (D) 
and x p (D) composed by k- and (n — fc)-tuples such that 
x s ; = [xf |x?], then the output of the n-in (n — k)-OMl encoder 
H T (L>) is s(D) = x(L>)H T (D) = x p (D) - p(D), where 
p(D) = x s (D)P(D) is the output from P(D) with x s (£>) 
as input; with this choice, the source encoder can re-use the 
same algorithm used in a systematic channel encoder. 

In case of turbo codes, up to symbol reordering, the 
channel encoder realizes a block-based transformation G = 
[Ijvfc|Po|Pi] where Pj has N(nj — k) outputs. If the source 
sequence (of N(no+rii — k) symbols) x is suitably broken into 
the sequences x s , x°, and x 1 composed by Nk, N(no—k), and 
N{ni — k) symbols, the systematic turbo-syndrome s = [s^s 1 ] 
can be evaluated as the difference between [x^x 1 ] and the 
turbo-parity corresponding to x s , obtained with a conventional 
turbo encoder. Each syndrome component s- 7 acts as a system- 
atic syndrome for the corresponding constituent code. 

Several algorithms have been proposed in literature for 
systematic syndrome decoding; if a side information sequence 
y(D) is present, they usually assume that there exists a mem- 
oryless correlation channel between x(Z?) and y(D) defined 
by p(yi\x.i). As shown below, despite they seem different, 
they actually obtain the exact same source reconstruction; this 
holds for both convolutional and turbo codes. 

A. Principal and Complementary Trellises 

In syndrome decoding, differently from channel decoding, 
the side information sequence must be decoded over the 
coset of the original code signalled by the syndrome. In (2) 
each other trellis section structure is modified for taking into 
account this fact. In particular, at time i, the actual structure 
is described by 

In practice, if Sj = the BCJR algorithm uses the principal 
(original) trellis, otherwise it uses one over q"~ k — l comple- 
mentary trellises that share the same state transitions of the 
original one but have different branch labels. Each input mes- 
sage /ij(x?) is modified too in order to take into account the 
known prior probability. More precisely, the input messages 

are /4( x ?) = Mi( x ?) and /4( x ?) = P( x i)M x i) K M x f |yf )■ 
The forward metrics and output messages are now 

^(o i ) = ^ x ^_ 1 ( x !, x l';i)c4-i(^-iK( x !K( x ?) 

A<(x?) = Dx?!. 1 (x? 1 x?;i)a;. 1 (<7 j _ 1 )^(a < ^W) . 

The output messages A^(x?) are also computed to estimate the 
most probable x^ as the one maximizing ^^(x^)A^(x^). 

B. Inverse Syndrome Formation 

Instead of modifying the trellis structure, in Q a codeword 
c(D) belonging to the signalled coset is formed and subtracted 



from the side informatiorQ prior to decoding over the original 
code. The source sequence equals x(Z?) = c(D) +cq(D) for 
some codeword cq(-D) belonging to the original code. Since 
y(D) - c(D) = c (.D) + [y(D) - x(L>)] and the code is 
tailored for the correlation channel (assumed linear), cq(D) 
can be in principle found decoding this difference; eventually, 
the sum with c(D) permits to reconstruct the source sequence. 

In practice, in correspondence of the (systematic) syndrome 
s(D) a suitable c{D) is the one such that Cj = [0|Sj]. Conse- 
quently, the actual input messages to the BCJR algorithm are 
<(x|) ocp(xf|y|-0) cx /4(x?) and^'(xf) oc p(x? |yf-s,); 
if the backward correlation channel (i.e. the one between the 
side information and the source) is linear, then /i"(x^) cx 
p(x? + 8 i |yf)ocK(x?+s i ). 

Despite the different strategy, the forward and the backward 
metrics are proportional to the ones evaluated in [2H For 
example, in fact, assuming a"_ 1 (fTj_ 1 ) cx a£_ 1 (oi_i)o 

oc^^_ 1 ( x ^ x f;i)aU(^-i)M;(<)M-( x ?) = «i(^) • 

Similarly, the translated output messages A'/(x| — 0) and 
A"(x^ — are proportional to A^(xf) and A^(x?), respec- 
tively, so that the sum of the MAP estimate that the algorithm 
finds for cq(-D) with c(D) coincides exactly with the MAP 
estimate evaluated in [2q 

C. Decoding Under Parity Perspective 

In ||4j, it is correctly observed that the syndrome re- 
ceived by the decoder is also a parity message. In particular, 
the systematic syndrome corresponds to the parity obtained 
with the systematic generator G'(D) = [1„ \P'(D)], where 
P'(-D) = H(D) T . The source is then recovered by jointly 
decoding the side information and the received parity message 
with the corresponding channel decoder. 

A realization of the system P'(-D) can be easily derived 
from a realization of P(-D). More precisely, its trellis section 
structure can be described by £% i _ 1 ('Xi,Si) = X<7*_i( x f) x ? — 
s^, where x| and x^ are a fc-tuple and an (n — fc)-tuple 
obtained from x, as above; ^^(x^s.;) indicates if at any 
time i the transition from state <Xj_x to state cr, is possible 
with the source n-tuple Xj as input and the syndrome (n— fc)- 
tuple Si as output. As a remark, in correspondence of each 
transition of the original trellis, this trellis has q n ~ k parallel 
transitions that share the same sub-label xf ; in practice, this 
trellis coincides with the source coding trellis of [2|. 

'Source and side information must be defined over the same alphabet. 

2 In this treatment, it is implied that all initial metrics Oq (<to) and /3j^(o"jv) 
(TV is the number of vectors) are the same, apart from some scaling factors. 

3 As xf takes on a finite group, J2~ x f( x > x f ) = J2~ x , Si f( x > x f ~ s 0- 

4 In (3), it is not clear if A"(xf) is actually maximized or rather a sub- 
optimal estimate of the parity portion ~xP(D) is found as the output of P(D) 
with the MAP estimate of the systematic portion x s (D) as input, in which 
case the resulting estimate of cq(D) would be a real codeword. 
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The BCJR algorithm takes now /x"'(xj) = p(xi)p(yj|x.j) oc 
p(xi|yf) oc ^(x|)^(x?) and /i"'(s<) = P( r i\ s 'i) as in P uts > 
where p(rj|sQ describes the memoryless channel between the 
syndrome sequence sent and the one received r(L>). Again, if 
this channel is error free, i.e. /i"'(sQ is non zero if and only if 

equals the actual syndrome Sj, the forward and the backward 
metrics involved in decoding equal the ones evaluated in 0. 
For example, in fact, assuming a^ 1 (<Ji_i) oc a' i _ 1 (ai-i), 

x ^Li( X i f ' X i _S i) Q! i-l( cr i-l)A t f ( x ^ 

oc^ X ^_ 1 (x| ) xf;iK_ 1 ( < 7 i _ 1 K(x|K(xf) = c4(a i ) . 

In addition, the MAP estimate for xf is found maximizing 

E-xjAtf'fcW'fc) * /4( x f)E~ X |/4( x ?)Ar( x *)> where 
the output message A"'(xj) satisfies 

Af ( Xi ) =^C*_ 1 (^> s D<-i(^-i)A'"(^)Mf &) 

~[*< |xf] 

Hence, this MAP estimate maximizes also /4(x| )A^(xf), i.e. it 
is the exact same estimate evaluated in [2|; the same can be 
shown for x^H 

D. Syndrome Trellis 

In (5) the source-to-syndrome realization implemented by 
the source encoder is expanded in order to construct a col- 
lection of q n ~ k syndrome trellises. In particular, each one of 
them is constructed using only the encoder state transitions that 
correspond to the release of a specific syndrome symbol. Then, 
during decoding, the trellis to be used at time i is specified 
by the corresponding syndrome symbol received s,. 

If the source encoder was a systematic syndrome generator, 
the source-to-syndrome realization could be derived as in the 
previous section, so that each syndrome trellis would be made 
of the same state transitions. At time i, with the input message 

M-'"( x i) = A4"( x *) k K( x i)/4( X D> the BCJR algorithm 
would use the trellis described by (xj; i) = (Xj, Sj). 
Consequently, the approach would be equal to the one de- 
scribed in the previous section. 

However, the fact that the source encoder considered in 
is not a systematic syndrome generator implies a substantial 
modification to the syndrome decoding process discussed here. 
Decoding based on the parity perspective as described above 
can as well handle non-systematic syndromes. A discussion 
about the advantages offered by this choice is out of the scope 
of this letter. The first consequence is that, in general, the trellis 

5 The slight performance increase shown in |4| w.r.t. the algorithm in (5) 
can be justified only assuming that a sub-optimal estimate for xP(D) is found 
in the latter (see note [4). 



over which the BCJR algorithm is eventually run no longer 
shares the trellis transitions used by the systematic channel 
encoder for the code w.r.t. which the syndrome is formed. 

IV. Maximum a Posteriori Probability Decoding 

The scheme based on inverse syndrome formation is 
the only one that exactly re-uses the trellis described by 
Xal-! ( x f ) X D' b ut requires that source and side information 
are defined over the same alphabet. Also, syndromes could be 
in general corrupted during transmission, but this is not always 
properly handled by the algorithms described above. 

If the systematic syndrome is received as r(Z?), 
the optimal MAP estimate can be simply found as 
argmax Xi p (xj|y(D)r(D)), i.e. marginalizing the function 
p (x(D)p(D)s(D)|y(-D)r(D)). Apart from some scaling fac- 
tors, this function is factorable into four terms, given by (i) the 
backward correlation channel p (x(D)|y(D)), (ii) the response 
X (p(D)\x. s (D)) of the parity encoder P(-D), (iii) the response 
£{s(D) = x p (D)— p(-D)} of the systematic syndrome former 
(£{•} indicates the condition in brackets), and (iv) the trans- 
mission channel p (r(D)\s(D)). If (i) and (iv) are memoryless, 
once (ii) is exploded into £ /*= ( d)\ IL X%\_ x ( x f > Pi) the 

V P(-D) I 

corresponding factor graph is the one shown in Fig. |l(b)| 

Since the resulting factor graph is a super-graph of the one 
in Fig. | l(a)| (and no additional cycles are added), in place of 
using an ad-hoc solution, syndrome decoding can re-use the 
same exact algorithm used in channel decoding. In particular, 
before running the BCJR algorithm, it is sufficient to form the 
correct input messages jii(pi) and, after, to properly process 
the output messages A(pj) in order to form the MAP estimate 
of x?. Under this perspective, it does not matter if r(D) and 
y(D) are defined over the same alphabet of x(Z?) or not. 

V. Conclusion 

It has been shown that several implementations of sys- 
tematic syndrome decoding lead actually to the same source 
reconstruction. The most straightforward and general way to 
achieve this estimate is to tackle the problem as a MAP 
one and use iterative message-passing in order to find (or 
approximate) the optimal solution. This permits to handle 
any correlation and transmission channels without the need 
of designing a new decoding algorithm. 
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